Skip to content

feat: import Origin 7.0552 OPJ worksheets#5

Open
Limdongcheng wants to merge 39 commits into
nmrtist:mainfrom
Limdongcheng:codex/origin-project-import
Open

feat: import Origin 7.0552 OPJ worksheets#5
Limdongcheng wants to merge 39 commits into
nmrtist:mainfrom
Limdongcheng:codex/origin-project-import

Conversation

@Limdongcheng

Copy link
Copy Markdown

Summary

  • add experimental, content-detected import for the fixture-proven classic Origin 7.0552 OPJ profile without installing, launching, or calling Origin
  • import supported worksheet numeric/text cells, nulls, column names, validated window/group names, project parameters, notes, and bounded diagnostics through the existing table preview flow
  • recognize CPYUA OPJU projects and reject them with a clear unsupported-format error without creating partial output
  • document the exact compatibility boundary in English and Simplified Chinese

Technical approach

  • plotx-io owns Origin probing, checked OPJ framing/record parsing, metadata association, cell decoding, resource budgets, and the engine-neutral project model
  • plotx-core preflights and converts supported worksheets into typed table snapshots without adding an execution-engine dependency to plotx-data
  • the desktop application probes by content, opens the selected path once, verifies the opened handle is a regular file, reuses that handle for classification and import, and presents all imported tables in the existing preview/confirm flow
  • the parser is an idiomatic Rust implementation based on the MIT-licensed OpenOPJ record descriptions with attribution; liborigin GPL source was not copied, translated, linked, or shipped

Actual support boundary

OPJ import

Supported only for the exact profile proven by the committed regression fixture:

  • classic CPYA 4.2673 552# framing with embedded Origin version 7.0552
  • f64, f32, signed i32, signed i16, fixed-width ASCII text, mixed numeric/text cells, nulls, and nonzero row offsets
  • validated Origin window/group names and column names
  • project parameters and notes as source metadata
  • one generated worksheet label, Sheet1, per supported window; this does not claim to recover original worksheet labels

An independently framed unsupported column or non-table object may be skipped only when its outer boundary is trusted, and each omission produces a warning. Ambiguous grouping, untrusted framing, corrupt structures, unsupported versions, and files with no safely importable worksheet fail closed.

OPJU recognition

CPYUA OPJU files are recognized by content, but no OPJU variant is imported in this PR. PlotX returns a clear unsupported message and creates no partial table or recent-file entry.

Known limitations

  • no graphs, formulas, scripts, analysis recomputation, executable saved analyses, matrices, embedded objects, preview images, OLE/XML extraction, or protected/encrypted projects
  • no verified support for non-ASCII text, long names, units, comments, column designations, dates, categorical values, code pages, other OPJ versions/profiles, or OPJU containers
  • compatibility is intentionally limited to one externally sourced real OPJ regression fixture plus synthetic boundary cases
  • owned-payload budgets do not count the small Arc control block or allocator-internal transient overhead

Safety and error handling

  • signature and initial-structure detection instead of extension-only dispatch
  • 128 MiB default input cap, checked one-byte oversize sentinel, fixed 16 KiB reads, fallible reservation, actual Vec capacity checks, and Vec to Arc payload-peak accounting
  • checked offsets, lengths, counts, arithmetic, parser/string/cell/metadata budgets, and bounded diagnostic work
  • same-handle reads, regular-file verification, and Unix O_NONBLOCK handling reduce path-replacement and FIFO blocking risks
  • no filesystem path extraction, archive extraction, decompression, script execution, or embedded object execution
  • truncation, corruption, invalid limits, unsupported versions/content, encoding failures, and allocation failures become structured errors or user-visible warnings rather than panics or silent data changes

Fixtures and licenses

  • test-origin-7.0552.opj: OpenOPJ test.opj, pinned source commit 42ddcf1eb3a490744c54fca0a4ed6fe7a5e723ca, MIT, 282,034 bytes, SHA-256 ac7f71c367562e85e9d4bb4ae418cbcaaa1b5dff80436180e8d3331c7e1d6308
  • RawData_Locust_Revision1_TIS_Mechanism.opju: Figshare DOI 10.6084/m9.figshare.28535426.v1, CC BY 4.0, 64,954 bytes, SHA-256 13c47a6a5daaf14493da59c8f1b284d9efb08129c8320b6ad9fd0b5191faa55f
  • both files are redistributed unchanged; source, attribution, license, byte length, and hashes are recorded beside the fixtures

Verification

  • cargo pr-check - passed all 7 stages on final HEAD: formatting, 800-line source limit, dependency licenses/advisories, default frontends, Clippy with warnings denied, all-feature/DataFusion tests, and reference-backend tests
  • cargo test -p plotx-io --locked origin - 100 Origin unit tests passed
  • cargo test -p plotx-io --locked --test origin_fixtures - 2 public fixture tests passed with concrete imported values/metadata assertions
  • cargo test -p plotx --locked ui::file_dialogs - 41 application import tests passed
  • pending-preview command regression test - passed
  • npm run build from docs/ - 63 English and Simplified Chinese pages built with Pagefind and sitemap
  • final independent read-only review - approved with no remaining Critical or Important findings

npm ci reported four advisories in the existing unchanged documentation lockfile (one moderate and three high). This PR does not update that toolchain or use npm audit fix --force.

Follow-up work

  • add more legally redistributable, independently verified OPJ versions and worksheet metadata profiles before expanding compatibility claims
  • reverse-engineer and implement a complete bounded OPJU outer-container profile before enabling OPJU import
  • add dedicated reader-adapter tests for short reads, finite Interrupted errors, and illegal over-reported byte counts

@github-actions

github-actions Bot commented Jul 22, 2026

Copy link
Copy Markdown

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

@Limdongcheng

Copy link
Copy Markdown
Author

I have read the CLA Document and I hereby sign the CLA

@Limdongcheng

Copy link
Copy Markdown
Author

recheck

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant